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ABSTRACT 



Achieving Change for Texans (ACT) was a demonstration 
program designed to help participants achieve independence from welfare 
through employment, training, temporary assistance, and support services. The 
ACT included the following four primary components: Time-Limited and 
Transitional Benefits (TL) ; Responsibilities, Employment, and Resources 
(RER) ; Incentives to Achieve Independence; and Temporary Assistance for Needy 
Families (TANF) One-Time Payments. At some RER sites (called Choice sites), 
welfare recipients agreed to a combination of RER and time limit provisions 
in return for an extra 6 months of transitional Medicaid and child care 
services. In June 1996-December 1997, a total of 39,116 welfare cases were 
assigned to either the control or experimental groups for one of the ACT 
components. Each component's early impacts were evaluated through a process 
evaluation, impact evaluation, and follow-up interviews. Very few differences 
in outcomes between the experimental and control groups were observed for the 
TL component. However, it appeared that the combination of RER and time limit 
provisions in the RER Choices experiment may be causing a statistically 
significant number of experimental group caretakers to leave and remain off 
TANF rolls in the early months of the experiment. (Twenty-four tables/figures 
are included. A technical appendix and additional statistical impacts are 
appended.) (MN) 
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Major Highlights of Research Findimgs 

This report includes net impacts of the Achieving Change for Texans (ACT) 
demonstration from June 1996 through December 1997, and describes early impacts of 
the Time Limits and Responsibilities, Employment and Resources (RER) interventions 
on welfare dynamics, labor market outcomes, participation in workforce development 
programs, and use of subsidized child care services. A total of 39,116 cases were 
assigned to either the experimental or control group for the ACT demonstration during 
the study period. 

Major highlights of the impact findings through December 1997 for each 
component of the demonstration are summarized below: 

• Time Limits 

Very few differences in outcomes between experimental and control groups were 
observed. Positive but very small net impacts were observed only for the transfer 
from TANF cash benefits to transitional benefits and the percentage of cases using 
subsidized child care each month. By the end of December 1997, only 92 persons 
had exited TANF in Bexar County because of reaching their time limit. 

Due to the length of time needed for time limits to take effect for Tier II and Tier 
111 recipients, more time needs to elapse before the true impacts of this experiment 
can be measured for these recipients. 

• Responsibilities, Employment and Resources 

Choices Research Sites. In Choices research sites, RER experimental group 
members spent a statistically significant smaller portion of time on TANF than their 
control group counterparts. Nearly 61 percent of caretakers in the experimental group 
exited from welfare compared to 58.2 percent for the control group. Penalty-related 
exits comprised 1 5 percent of all exits for the experimental group. Members of the 
experimental group also experienced a statistically significant lower rate of welfare 
recidivism in the year following exit (34.6 percent) than control group members (38.7 
percent). No statistically significant differences in employment rates or earnings 
between the two groups were observed. 

Taken together, these findings suggest that the combination of RER and time 
limits provisions in the RER Choices experiment may be causing a statistically 
significant number of experimental group caretakers to leave and remain off the 
TANF rolls in the early months of the experiment. However, there is as yet no 
evidence, based on U1 wage records, that these persons are becoming employed at 
higher rates. 

Non-Choices Sites. In Non-Choices sites, welfare dynamics results for the first year 
of the RER experiment indicated that experimental group members spent a 
statistically significant smaller portion of time on TANF than their counterparts in the 
control group. No significant differences were found for any of the other welfare exit 
measures. Also, no significant differences between the experimental and control 
groups were found for employment, earnings, or any of the child care measures. 

Primarily due to the small sample and the fact that the data include only early 
months of the Non-Choices experiment, it would be premature to infer much from the 
interim findings for this experiment at this time. 



Executive Summary 



In 1995, the Texas Legislature enacted H. B. 1863, which formed the basis for 
Texas’ waiver from existing Federal laws governing the Aid to Families with Dependent 
Children (AFDC) program.' The Texas waiver, officially known as the Achieving 
Change for Texans (ACT) demonstration, aims to assist participants to achieve 
independence from welfare through an increased emphasis on employment, training, 
temporary assistance and support services. It includes four primary components: 

o Time-Limited and Transitional Benefits (TL) 

. Responsibilities, Employment and Resources (RER) 

• Incentives to Achieve Independence (lAI) 

® TANF One Time Payments 

The ACT demonstration was first implemented in June 1 996 and is scheduled to continue 
operating through March 2002. The Texas Department of Human Services (DHS) is 
responsible for implementation and oversight of the ACT demonstration. 

The evaluation of the ACT demonstration consists of three approaches: a process 
evaluation, conducted by the DHS Office of Program Analysis and Evaluation; an impact 
analysis, conducted by the Center for the Study of Human Resources (CSHR) of the LBJ 
School of Public Affairs at the University of Texas at Austin; and follow-up interviews, 
conducted by the School of Social Work at the University of Texas at Austin, with 
persons who reached their time limits or who elected to receive TANF One Time 
payments instead of entering TANF. 

This report includes net impacts of the ACT demonstration from its inception in 
June 1996 through December 1997, and describes early impacts of the time limits and 
RER experiments on welfare dynamics, client self-sufficiency, participation in workforce 
development programs, and use of subsidized child care services.^ Later reports will 
extend the period for which net impacts are computed, and also will measure the impact 



‘ Since the passage of H.B. 1863, the AFDC program has been replaced by a new cash assistance program 
called Temporary Assistance For Needy Families (TANF). This report will use the term, TANF, to refer to 
Texas’ cash assistance program for welfare recipients. 

^ The TANF One-Time payment component is not included in the impact analysis because it was not 
implemented as a randomized experiment. Implementation of Incentives to Achieve Independence (lAI) 
has not yet occurred. 
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of the demonstration on children, including their education, immunization, and need for 
child protective services. 

Demonstration Components Included in this Analysis 
Time-Limited and Transitional Benefits 

This statewide initiative limits the number of months that able-bodied adult 
caretakers can receive TANF benefits. It provides 12 months of transitional Medicaid 
and child care for all persons reaching their time limits and 1 8 months to persons who 
voluntarily participate in the Choices program. 

The time limit placed on each caretaker’s TANF benefits is based on his/her work 
history and educational attainment. The most job-ready clients (Tier I) are eligible for up 
to 12 months of cash assistance after notification of an opening in the Choices program\ 
Less job-ready clients (Tier II) may receive up to 24 months of benefits after Choices 
notification, and the least job-ready clients (Tier III) may receive as many as 36 months 
of cash assistance .“ 

Adult clients who exhaust their time-limited benefits are disqualified from 
receiving TANF for five years. Exemptions to this freeze-out period are granted if local 
economic conditions or severe personal hardships exist which prevent the client from 
remaining independent of TANF. 

The time limits experiment is being evaluated in Bexar County. Adult caretakers 
assigned to the experimental group are subject to both time limits and RER provisions 
while control group members must only meet RER requirements (see below). 
Implementation of this experiment began in June 1996. 

Responsibilities, Employment, and Resources in Choices Counties 

The Personal Responsibility Agreement (PRA) is a central feature of RER that 
requires adult TANF applicants and recipients to comply with specific responsibilities as 
a condition of TANF eligibility. Responsibilities include: 



’ The Choices program has replaced the Texas JOBS program. 

" The 36-month “clock” does not begin “ticking” until twelve months after an in-depth assessment has been 
completed for Tier 111 individuals who are offered an opportunity to participate in the Texas Choices 
program. 



e compliance with Choices program participation requirements; 

0 cooperation with child support and paternity establishment efforts; 

. completion of regularly-scheduled Texas Health Steps screenings for children;’ 
o compliance with immunization requirements for pre-school children; 

B compliance with school attendance policies for children; and 
s participation in parenting skills training classes if referred. 

Under the PRA, parents also must: 

. not voluntarily quit a paying job of at least 30 hours per week; and 
. refrain from selling or abusing illegal or controlled substances or abusing alcohol. 

Clients who fail to comply with PRA requirements without good cause will 
receive a finaiicial penalty. Failure to comply with Choices participation or child support 
cooperation requirements results in a $78 dollar penalty per month of non-compliance for 
single-parent families, and a $125 per month penalty for two-parent families. Failure to 
comply with other PRA requirements carries a $25 per month per non-compliance 
penalty, with a maximum penalty of $75 per month. 

Other RER provisions include the disregard of children’s earnings and resources 
in the calculation of family benefits, increased resource limits permitted for eligibility 
determination, and the elimination of the work history requirement and 100-hour work 
rule for TANF-Unemployed Parent (TANF-UP) families. 

RER is being evaluated in four offices in Choices counties (Beaumont, Odessa, 
the Dillon office in Corpus Christi, and the Clint office in El Paso). Persons in these 
locations who are assigned to the experimental group must comply with both RER and 
time limit provisions. Neither of these welfare reform provisions are applicable to 
control group members, who were subject to sanctions under the pre-welfare-reform 
rules. RER in these sites was implemented in June 1996, with time limit provisions 
added in January 1997. 



’ Texas Health Steps (THSteps) is the Texas version of the Medicaid program known as Early and Periodic 
Screening, Diagnosis and Treatment (EPSDT) Program. 

® While the Clint office is located in a Choices county, it does not currently offer Choices services due to 
remoteness. Because of this difference from other RER-Choices sites, results from the Clint office are not 
reported with other RER-Choices results. 



Responsibilities, Employment, and Resources in Non-Choices 
Counties 

Many rural counties in the state cutrently do not offer Choices services to TANF 
recipients. The RER experiment is being evaluated in four offices in Non-Choices 
counties (Hondo, Huntsville, Lockhart and Luling). RER provisions are identical to those 
described above except for those related to Choices participation. In these sites, 
experimental group members are subject to RER provisions while control group members 
do not have to meet RER requirements, but were subject to pre-welfare-reform rules. 
Because time limits are tied to an offer of participation in the Choices program, neither 
group is subject to time limit provisions. RER in Non-Choices counties was 
implemented in January 1997. 

Research Questions 

Each of the treatments is expected to influence welfare recipients’ behavior in the 
areas of welfare dynamics, client self-sufficiency, participation in workforce 
development programs, education and immunization of children, the use of subsidized 
child care and child protective services. A number of variables for each of these research 
questions has been developed to measure the demonstration’s effect on these measures. 

Methodological Approaches 
Data Sources 

The research data set for this evaluation was created by linking a number of 
administrative data files on program participation, demographic characteristics, and 
outcomes at the individual and case levels across agencies, and programs and over time. 
This report includes data collected from the Texas Department of Human Services (DHS) 
and the Texas Workforce Commission (TWC). Data from most sources were collected 
from June 1994, two years prior to the beginning of the ACT demonstration, through 
December 1997. 

Data needed to measure education, immunization, and child protective services 
outcomes were not available for this report. Net impacts on those measures will be 
incorporated into future reports as the requisite data become available. 
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Statistical Methods 

The impact analysis utilizes an experimental research design in which persons 
were randomly assigned to experimental and control groups when they either began 
receiving TANF benefits or were recertified for benefits. Most of the analyses were 
performed using data covering the entire experimental and control populations. For some 
research questions, additional analyses were conducted for subgroups of the population. 

Tests of random assignment were conducted to verify that any differences 
between the experimental and control groups occurred only by chance. After 
determining that each experiment passed the tests of random assignment, unadjusted net 
impacts were computed by comparing the differences between the means of the 
experimental and control groups. These results were then adjusted to account for 
demographic differences between the groups to produce adjusted net impacts. 

Potential Limitations of Analysis 

Some of the factors that could limit the usefulness of this analysis include: 

. The short time from the beginning of the demonstration to the date of this report; 

e Different lengths of time available to follow evaluation participants due to their 
entering the evaluation in different time periods; 

o The inability to obtain some of the administrative data files needed to analyze 
several of the research questions; and 

• Possible contamination of treatments caused by caseworkers, by word of mouth or 
news reports, or by other agencies’ non-experimental treatments. 

Most of these limitations can be overcome by allowing more time to elapse prior to 

completing this analysis. 

Summary of Findings from Each Experiment 

From June 1996 through December 1997, a total of 39,1 16 cases were assigned to 
either the control or experimental group for one of the ACT components. Impact 
findings through December 1997 for each component of the demonstration are 
summarized below. 




12 



Time Limits 



The time limit experiment was implemented in June 1996. By the end of 
December 1997, 25,552 cases were assigned to participate as either experimental or 
control group members. Test of random assignment revealed no statistically significant 
differences between the two groups. 

Very few differences in outcomes between experimental and control groups in the 
time limits experiment were observed for the period of study included in this report. 
After analyzing welfare dynamics, self-sufficiency, participation in workforce 
development services, and use of subsidized child care, positive net impacts were 
observed only for transfer from TANF cash benefits to transitional benefits and the 
percentage of cases using subsidized child care each month.’ The magnitude of both of 
these impacts were quite small (0.9% for transfer to transitional benefits and 0.25% for 
cases using subsidized child care monthly). Approximately 1 3 percent of all exits for 
both the experimental and control groups occurred in the month of or the month 
following the assessment of a penalty for failure to comply with PRA provisions. 

By the end of December 1997, only 92 persons had exited TANF in Bexar County 
because of reaching their time limit. Due to the length of time needed for time limits to 
take effect for Tier II and Tier III recipients, more time needs to elapse before the true 
impacts of this experiment can be measured for these recipients. 

RER Choices Experiment 

From June 1996 through December 1997, 9,566 cases were assigned to participate 
in the RER Choices experiment as either experimental or control group members. Tests 
of random assignment revealed no statistically significant differences between the two 
groups. 

In the first 19 months of the RER experiment in Choices counties, a statistically 
significant difference in the proportion of time spent on welfare was observed, with the 
experimental group spending less time on TANF than their counterparts in the control 
group. There was also a statistically significant difference in the rate of transfer to 

’ Transitional benefits include cases in program types 7, twelve months post MEDICAID resulting from 
increase in earnings or combined increase in earnings and child support, 20, four months post MEDICAID 
coverage resulting from increase in child support, 29, twelve or eighteen months transitional MEDICAID 
after expiration of TANF time-limited, and 37, twelve months post MEDICAID coverage resulting from 
loss of 30+1/3 disregard. 



medical assistance-only status, transitional benefits-only status and to payee-only status. 
The experimental group had higher rates of transfer to medical assistance-only and 
transitional benefits-only status, and lower rates of transfer to payee-only status, relative 
to the control group. These patterns contribute to the finding of a relatively small 
difference in the overall caretaker exit rates for the two groups, with 60.8 percent of 
caretakers in the experimental group exiting welfare compared to 58.2 percent for the 
control group. Penalty-related exits* comprised 15 percent of all exits for experimental 
group members. 

A statistically significant difference in the rate of welfare recidivism in the year 
following exit was observed between the experimental and control groups, with the 
experimental group members experiencing a lower rate (34.6 percent) than control group 
members (38.7 percent). Although experimental group members left and remained off 
TANF at higher rates than control group members, there were no statistically significant 
differences in employment rates or earnings between the two groups, based on earnings 
data from UI wage records. A small but statistically significant difference in the rate of 
Choices participation was observed, with members of the experimental group having 
lower Choices participation rates than control group members due to lower levels of 
participation in education and job search activities. 

Finally, although there was no statistically significant difference in the rates of 
child care usage between the two groups, there was a statistically significant difference in 
the amount of subsidy per month. The experimental group members used a larger 
amount of child care subsidy ($199) per child-month than their counterparts in the control 
group ($190). This finding, when coupled with the finding of higher transfers from cash 
assistance to transitional benefits, suggests that experimental group members may be 
using the larger child care subsidy to assist in their transition from welfare to work. 

Taken together, these findings suggest that the combination of RER and time 
limits in the RER Choices experiment may be causing a number of caretakers to leave 
and remain off the TANF rolls in the early months of the experiment. However, based on 
UI wage records, there is no evidence as yet that these persons are becoming employed at 



* For the purpose of this report, penalty-related exits are defined as exits from TANF accompanied by a 
penalty in the same month or prior month. 
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higher rates. These measures, along with additional measures of child well-being, will 
continue to be examined in future reports. 

RER Non-Choices Experiment 

The RER experiment in Non-Choices research sites was implemented in January 
1997. By the end of December 1997, 1,358 cases were assigned to participate as 
experimental or control group members. While no statistically significant differences 
were observed between the sizes of the two groups, there are some differences in the 
demographic characteristics of the groups. 

Analysis of welfare dynamics for the first year of the RER experiment in Non- 
Choices counties showed a statistically significant difference in the proportion of time 
spent on TANF between the experimental and control groups, with experimental group 
members spending less time on TANF than their counterparts in the control group. 
However, no statistically significant differences were found for any of the other welfare 
exit measures. The inconsistency between results for the proportion of time spent on 
TANF and other welfare exit measures is probably due to the relatively short time period 
available in which to calculate welfare exits and the small sample size of participants in 
this experiment. Approximately five percent of all exits were penalty-related. As of 
December 1 997, insufficient time had elapsed to calculate welfare recidivism rates for 
this experiment. 

No statistically significant differences between the experimental and control 
groups were found for employment, earnings, or any of the child care measures. A 
statistically significant difference in the rate of JTPA participation was observed, with 
experimental group members having lower JTPA participation rates than control group 
members. While it was statistically significant, the magnitude of the difference between 
the two groups was not very large — amounting to less than 0.5 percentage points.’ 



’ Statistical procedures have been applied to determine whether observed differences between the control 
and experimental group are due to chance or are a consequence of the experimental treatments. If the 
difference is large enough that it may not be attributed to chance, then ’the difference is said to be 
“statistically significant.” Because of the large sample sizes used in some of the experiments, the statistical 
estimates of the experimental effect are unusually precise relative to most social science research. Because 
of this great precision, it is often the case that an estimated difference may be “statistically significant” but 
still be quite trivial. 



Primarily due to the limited time period included in this report and the small 
sample size in this experiment, it; would be premature to infer much from the interim 
findings for this experiment. 

Future Reports 

Future evaluation reports on the ACT demonstration will report impacts on all of 
the measures included in this report, as well as impacts of the demonstration on various 
children’s outcomes. The final report, due in May 2002, will include impacts through 
December 2001. 
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L Overview 



In 1995, the Texas Legislature enacted H. B. 1863, which formed the basis for 
Texas’ waiver from existing Federal laws governing the Aid to Families with Dependent 
Children (AFDC) program.'® The Texas waiver, officially known as the Achieving 
Change for Texans (ACT) demonstration, aims to assist participants to achieve 
independence from welfare through an increased emphasis on employment, training, 
temporary assistance and support services. The demonstration, designed to test a number 
of policy provisions to reduce dependence, encourage personal responsibility, and 
increase savings, includes four primary components: 
o Time-Limited and Transitional Benefits (TL) 

« Responsibilities, Employment and Resources (RER) 

. Incentives to Achieve Independence (lAI) 
o TANF One-Time Payments 

The ACT demonstration was first implemented in June 1996 and is scheduled to continue 
operating through March 2002. The Texas Department of Human Services (DHS) is 
responsible for implementation and oversight of the ACT demonstration. 

The evaluation of the ACT demonstration consists of three approaches: a process 
evaluation, which is being conducted by the DHS Office of Program Analysis; an impact 
analysis, which is being conducted by the Center for the Study of Human Resources 
(CSHR) of the LBJ School of Public Affairs at the University of Texas at Austin; and 
follow-up interviews, conducted by the School of Social Work at the University of Texas 
at Austin, with persons who reached time limits and elected to receive TANF One Time. 

This report includes net impacts of the ACT demonstration from its inception in 
June 1996 through December 1997, and describes early impacts of the time limits and 
RER experiments on welfare dynamics, client self-sufficiency, participation in workforce 
development programs, and use of subsidized child care services". Later reports will 



'® Since the passage of H.B. 1 863, the AFDC program has been replaced by a new cash assistance program 
called Temporary Assistance For Needy Families (TANF). This report will use the term, TANF, to refer to 
Texas’ cash assistance program for welfare recipients. 

" The TANF One-Time payment component is not included in the impact analysis because it was not 
implemented as a randomized experiment. Implementation of Incentives to Achieve Independence (lAI) 
has not yet occurred. 



er|c 



1 



17 



extend the period for which net impacts are computed, and also will measure the impact 
of the demonstration on children, including their education, immunization, and need for 
child protective services. 

2. Impact Analysis 

The purpose of the impact analysis is to measure the net impact of the Time- 
limited and transitional benefits (TL) and Responsibilities, Employment, and Resources 
(RER) components of the Achieving Change for Texans (ACT) demonstration on a 
number of measures of client well-being. To accomplish this objective, participants 
were randomly assigned to experimental and control groups in each of the research sites 
for the TL and RER demonstration components. Different sites were chosen to measure 
the effects of RER in both Choices and Non-Choices locations. 

2.1 Demonstration Components Being Evaluated 

2.1.1 Time-Limited and Transitional Benefits 

This statewide initiative limits the number of months that able-bodied adult 
caretakers can receive TANF benefits. It provides 12 months of transitional Medicaid 
and child care for all persons reaching their time limits and 18 months to persons who 
voluntarily participate in the Choices program. 

The time limit placed on each caretaker’s TANF benefits is based on his/her work 
history and educational attaiiunent. The most job-ready clients (Tier I) are eligible for up 
to 12 months of cash assistance after notification of an opening in the Choices program. 
Less job-ready clients (Tier II) may receive up to 24 months of benefits after Choices 
notification, and the least job-ready clients (Tier III) may receive as many as 36 months 
of cash assistance.” 

Adult clients who exhaust their time-limited benefits are disqualified from 
receiving TANF for five years. Exemptions to this freeze-out period are granted if local 



” The Choices program has replaced the Texas JOBS program. 

” The 36-month “clock” does not begin “ticking” until twelve months after an in-depth assessment has 
been completed for Tier 111 individuals who are offered an opportunity to participate in the Texas Choices 
program. 
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economic conditions or severe personal hardships exist which prevent the client from 
remaining independent of TANF. 

The time limits experiment is being evaluated in Bexar County. Adult caretakers 
assigned to the experimental group are subject to both time limits and RER provisions 
while control group members must only meet RER requirements. Implementation of this 
experiment began in July 1996. 

2.1.2 Responsibilities, Employment, and Resources in Choices Counties 

The Personal Responsibility Agreement (PRA) is a central feature of RER that 
requires adult TANF applicants and recipients to comply with specific responsibilities as 
a condition of TANF eligibility. Responsibilities include: 

. compliance with Choices program participation requirements; 

• cooperation with child support and paternity establishment efforts; 

• completion of regularly-scheduled Texas Health Steps screenings for children;''' 

. compliance with immunization requirements for pre-school children; 

o compliance with school attendance policies for children; and 
. participation in parenting skills training classes if referred. 

Under the PRA, parents also must; 

• not voluntarily quit a paying job of at least 30 hours per week; and 

o refrain from selling or abusing illegal or controlled substances or abusing alcohol. 

Clients who fail to comply with PRA requirements without good cause will 
receive a financial penalty. Failure to comply with Choices participation or child support 
cooperation requirements results in a $78 dollar penalty per month of non-compliance for 
single-parent families, and a $125 per month penalty for two-parent families. Failure to 
comply with other PRA requirements carries a $25 per month per non-compliance 
penalty, with a maximum penalty of $75 per month. 

Other RER provisions include the disregard of children’s earnings and resources 
in the calculation of family benefits, increased resource limits permitted for eligibility 



Texas Health Steps (THSteps) is the Texas version of the Medicaid program known as Early and 
Periodic Screening, Diagnosis and Treatment (EPSDT) Program. 



determination, and the elimination of the work history requirement and 100-hour work 
rule for TANF-Unemployment Parent (TANF-UP) families. 

RER is being evaluated in four offices in Choices counties (Beaumont, Odessa, 
the Dillon office in Corpus Christi, and the Clint office in El Paso).'* Persons in these 
locations who are assigned to the experimental group must comply with both RER and 
time limit provisions. Neither of these welfare reform provisions are applicable to 
control group members. RER in these sites was implemented in June 1996, with time 
limit provisions added in January 1997. 

2.1.3 Responsibilities, Employment, and Resources in Non-Choices Counties 

Many rural counties in the state currently do not offer Choices services to TANF 
recipients. The RER experiment is being evaluated in four offices in such counties 
(Hondo, Huntsville, Lockhart and Luling). RER provisions are identical to those 
described above except for those related to Choices participation. In these sites, 
experimental group members are subject to RER provisions while control group members 
do not have to meet RER requirements. Because time limits are tied to an offer of 
participation in the Choices program, neither group is subject to time limit provisions. 
RER in Non-Choices counties was implemented in January 1997. 

2.2 Research Questions 

Each of the treatments is expected to influence welfare recipients’ behavior in the 
areas of welfare dynamics, client self-sufficiency, participation in workforce 
development programs, education and immunization of children, and the use of 
subsidized child care and child protective services. The net impacts of each experiment 
on these outcomes will be measured, with the expected directions of the impacts 
summarized in Table 1. 



While the Clint office is located in a Choices county, it does not currently offer Choices services due to 
remoteness. Because of this difference from other RJER-Choices sites, the Clint office was analyzed as a 
separate experiment. Results are included in Appendix B. 
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Table 1. Summary of Expected Treatment Impacts for Outcomes 





Treatments 


Outcomes 


TL 


RER in 
Choices 


RER in 
Non- 
Choices 


Promote self-sufficiency 


+ 


+ 


+ 


Promote education and immunization 
of children 








Welfare dynamics 


+ 


+ 


+ 


Access to subsidized child care 


? 


+ 


+ 


Participation in workforce 
development services 


? 


+ 




Child protective services 









Key: “+” means the treatment is expected to have a positive effect on the outcome. 

means the treatment is expected to have a negative effect on the outcome. 

“?” means the treatment is expected to have an effect on the outcome, but the direction is 
ambiguous. 

Blank cell means the treatment is not expected to have an effect on the outcome. 

Shaded cell means the data were not available for this report. 

2.3 Methodological Approaches 

2.3.1 Specific Variables to be Analyzed and Data Sources Used 

For each of the outcomes identified above, a set of variables was created from 
existing administrative data files to measure the effects of the ACT demonstration. These 
are summarized in Table 2. Variables and data sources used to create them are described 
more fully in Appendix A. 

Data from most sources were collected from June 1994, two years prior to the 
beginning of the ACT demonstration, through December 1997'*. CSHR researchers are 
still negotiating with several agencies to obtain the data files needed to measure 
education, immunization, and child protective services outcomes. Data from other 
sources, including child support and higher education, were not received in time to be 
included in this report. Results from these data sources will be incorporated into future 
reports as they become available. 



'* For this report, child care data files were only available through June 1997. 
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Table 2. Specific Variables Analyzed 



Outcomes 


Variables to be Analyzed 


Data Source 


Promote self- 
sufficiency 


Employment, earnings, total family 
income, child support collections 


Department of Human 
Services 

administrative data, 
Texas Workforce 
Commission UI wage 
data. Office of 
Attorney General data 


Promote education 
and immunization 
of children*’ 


School attendance, immunization, 
penalties imposed for failure to 
obey school attendance and 
immunization provisions. 


Department of Human 
Services 

administrative data, 
Texas Education 
Agency Public 
Education Information 
Systems Management 
System (PEIMS) data, 
Texas Department of 
Health ImmTrac data 


Welfare dynamics 
(entries, exits, 
recidivism) 


Administrative data regarding: 
TANF spells, penalties, time-limit 
clocks 


Department of Human 
Services 

administrative data 


Subsidized child 
care 


Subsidized child care spell dates 
and number of children covered 


Department of Human 
Services child care 
payment data, Texas 
Workforce 
Commission Child 
Care Management 
System data 


Participation in 
workforce 
development 
services 


Choices scheduled and actual 
participation hours by component 
activity; JTPA; post-secondary 
participation data 


Texas Workforce 
Commission, Texas 
Higher Education 
Coordinating Board 
program participation 
data 


Child protective 
services** 


Child abuse and neglect (number of 
substantiated investigations by 
protective agency); foster care 
placement 


Child Protective 
Services data 



2.3.2 Statistical Methods 

The impact analysis utilizes an experimental research design in which persons 
were randomly assigned to experimental and control groups when they either began 
receiving TANF benefits or were recertified for benefits. Most of the analyses were 



” Data to measure these outcomes were not available for this report. 
Data to measure these outcomes were not available for this report. 
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performed using data covering the entire experimental and control populations. For some 
research questions, additional analyses were conducted for subgroups of the population. 
In analyzing time limits, for example, it ,is necessary to group the population by time- 
limit tiers because each tier is expected to respond differently to time limits. 

In a random experiment, the characteristics of the experimental and control 
groups should differ only by chance.” However, tests of random assignment were 
conducted to identify any statistically significant differences remaining in either the 
number of persons assigned to the experimental and control groups or the characteristics 
of persons in each group. Any differences between the groups were investigated for their 
potential for biasing statistical impacts. 

After determining that each experiment passed the tests of random assignment, 
unadjusted net impacts were computed by comparing the differences between the means 
of the experimental and control groups. These results were then statistically adjusted to 
account for demographic differences between the groups to produce adjusted net 
impacts.^ 

2.3.3 Potential Limitations of Analysis 

Some of the factors that could limit the usefulness of this analysis include; 

o The short time from the beginning of the demonstration to the date of this report; 

. Different lengths of time available to follow evaluation participants due to their 
entering the evaluation in different time periods; 

o The inability to obtain some of the administrative data files needed to analyze 
several of the research questions; and 

• Possible contamination of treatments caused by caseworkers, by word of mouth or 
news reports, or by other agencies’ non-experimental treatments.^' 



” Statistical procedures have been applied to determine whether observed differences between the control 
and experimental group are due to chance or are a consequence of the experimental treatments. If the 
difference is large enough that it may not be attributed to chance, then the difference is said to be 
“statistically significant.” Because of the large sample sizes used in some of the experiments, the statistical 
estimates of the experimental effect are unusually precise. Because of this great precision, it is often the 
case that an estimated difference may be “statistically significant” but still be quite small. 

The adjustment is accomplished by means of linear regression. The dependent variable for each 
regression is the outcome for which the impact is being estimated. The independent variables include a 
dummy variable equal to one if the case was in the experimental group and zero otherwise, and a set of 
demographic descriptors such as age, race and education. The regression coefficient for the experimental 
group dummy variable estimates the adjusted net effect of the experimental treatment. 

These limitations are discussed more fully in Appendix A. 
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Some of these limitations can be overcome simply by having more time to observe 
outcomes for evaluation subjects. CSHR and DHS are continuing negotiations with 
several state agencies to obtain the additional administrative data needed for this analysis. 
As new and potentially contaminating events occur, CSHR researchers are reviewing 
these to verify that any contamination falls equally upon experimental and control 
groups. 

3. Estimated Statistical Impacts To Date 

From June 1996 through December 1997, a total of 39,1 16 subjects were assigned 
to either the experimental or control group for one of ACT components. Table 3 provides 
sample sizes for each group by experiment. 



Table 3. Number of Subjects Assigned 





Experiment 


Time Limits 


RER Choices 


RER Non- 
Choices 


Clint 


Total 


Time period of 
analysis (from/to) 


07/96 12/97 


07/96 12/97 


1/97 12/97 


07/96 12/97 


Experimental 


12,735 


4,848 


693 


1,420 


19,696 


Control 


12,817 


4,718 


665 


1,220 


19,420 


Total 


25,552 


9,566 


1,358 


2,640 


39,116 



Impacts for the time limits, RER-Choices and RER Non-Choices components of 
the ACT demonstration through December 1 997 are summarized below. Due to the short 
time period that elapsed from the begirming of the demonstration through the end of the 
period included in this report, many of these short-term results may be influenced by 
persons who were already receiving TANF at the begirming of the demonstration and 
may change in subsequent analyses that are based on more complete data. 

3.1 Time Limits Experiment 

From June 1996 through December 1997, 25,552 cases were assigned to 
participate in the time limits experiment as either experimental or control group 



members. As can be seen in Figure 1, most of these cases were already receiving TANF 
at the beginning of the experiment. Tests of random assignment revealed no statistically 
significant differences in either the number of persons assigned to experimental or control 
groups or in the demographic characteristics of the caseheads (Table 4). Thus, for the 
time period included in this report, the time limits experiment passed all tests of random 
assignment. 



Figure 1. Number of Cases Assigned to Time Limits (by month) 
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Table 4. Time Limits Experiment: 
Characteristics of Experimental and 
Control Group Members at Random Assignment 



Attribute 


Experimental 


Control 


Experimental- 

Control 

Difference 


Did not finish high 
school 


21.2% 


21.6% 


-0.4% 


Male 


5.8% 


6.0% 


-0.2% 


Black 


13.5% 


14.1% 


-0.6% 


Hispanic 


73.2% 


72.3% 


0.9% 


Age (in years) 


30.8- 


30.7 


0.1 


Months on TANF out 
of last 12 months 


6.8 


6.8 


0.0 


Months employed out 
of last 12 months 


3.6 


3.6 


0.0 


Any employment in 
last 12 months 


48.5% 


48.3% 


0.2% 


Total U1 earnings in 
last 12 Months 


$1,806 


$1,839 


$33 


In TANF UP at time 
of random assignment 


3.1% 


3.1% 


0.0% 



None of the experimental-control differences were statistically significant 



Net impacts of the time limit experiment on welfare dynamics, self-sufficiency, 
participation in workforce development services, and use of subsidized child care are 
discussed below.” 

3.1.1 Welfare Dynamics 

3.1. 1.1 Welfare Exits and Recidivism 

Table 5 shows that there were no statistically significant differences between 
participants in the experimental and control groups in total exit rate, or transfers from 
TANF cash assistance to medical assistance-only and payee-only status. However, 
statistically significant differences were observed in the rate of transfer to transitional 
benefits only status, with members of the experimental group transferring to transitional 
benefits-only status at a higher rate than their control group counterparts.” 



A description of how each statistical measure was calculated is included in Appendix A. 

” Transitional benefits include cases in program types 7, twelve months post MEDICAID resulting from 
increase in earnings or combined increase in earnings and child support, 20, four months post MEDICAID 
coverage resulting from increase in child support, 29, twelve or eighteen months transitional MEDICAID 
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The table also shows the recidivism rates for the time limits experiment. No 
statistically significant difference between the experimental and control groups were 
observed for either one-year recidivism rates or average time to recidivism. 



Table 5. Welfare Exits and Recidivism for Time Limits 





Post-Treatment Mean 


Experimental 

-Control 


Adjusted 

Net 




Experimental 


Control 


Difference 


Impact 


Proportion of time spent on TANF out 
of maximum possible 


62.9% 


61.3% 


-0.2% 


-0.2% 


Exits 






Total caretaker exit rate 


51.6% 


52.1% 


-0.7% 


-0.5% 


Rate of transfer to medical 
assistance only status 


11.0% 


10.4% 


0.6% 


0.6% 


Rate of transfer to transitional 
benefits only status 


11.2% 


10.5% 


0.7%* 


0.9%** 


Rate of transfer to payee-only or 
case-name only status 


6.7% 


6.5% 


0.2% 


0.3% 


Recidivism^* 






One-year recidivism rate for all 
exits 


28.4% 


28.8% 


-0.4% 


-0.4% 


Average months to recidivism for 
exits followed by recidivism 


2.27 


2.29 


-0.02 


-0.01 



** statistically significant at 5% level 
* statistically significant at 10% level 



3.1. 1.2 Time-Limit Induced Exit Rates 

One of the expected outcomes of the time limits component of the experiment 
was that some of the participants on whom time limits provisions were imposed would be 
forced to leave the welfare rolls when their time limits had been reached. The magnitude 
of this impact can be estimated by tabulating the number of forced exits and the rate of 
forced exits out of the total caseload. By December 1997, only 92 exits occurred among 
Tier 1 clients due to their reaching their time limit. This accounted for 1 .4 percent of all 
Tier 1 cases. Because only 19 months had elapsed between the beginning of the 



after expiration of TANF time-limited, and 37, twelve months post MEDICAID coverage resulting from 
loss of 30+ 1 /3 disregard. 

Payee-only and case-name only status covers those adult recipients whose needs are no longer included 
in (dropped off from) the TANF cash grants but their children remain on active cases. 

“ Federal project reviewers have suggested that there may be possible bias in the recidivism estimates due 
to self-selection. Future reports will discuss the possibility of self-selection bias in more depth. 
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experiment and the end of the period covered by this report, no Tier II or Tier III clients 
were forced to exit TANF due to the expiration of their time limits. 

3.1. 1.3 Penalty Rates 

In the time limits experiment, both experimental and control group members were 
subject to financial penalties for failure to comply with provisions of the PRA.“ Table 6 
indicates that approximately eight percent of case months were spent in penalty status for 
both experimental and control group members through December 1997. While 
statistically significant differences between the two groups were observed in child 
support, drug abuse, and 2"‘* offenses in the Choices program, the magnitude of the 
differences was quite small. The average length of completed penalties is slightly less 
than three months for each group. 

For the purpose of this report, penalty-related exits are defined as exits from 
TANF accompanied by a penalty in the same month or prior month. As shown in Table 
6, penalty-related exits comprised approximately 13 percent of all exits for both 
experimental and control group members. No statistically significant differences were 
found between the two groups for this measure. 



The variable used to measure penalties in this report records each identified penalty and its scheduled 
duration. Data for the actual amount of financial penalties assessed will be included in lliture reports. 



Table 6. Penalties Identified in Time Limits Experiment 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percent of case-months in penalty status 


School Attendance 


2.79% 


2.76% 


0.03% 


0.03% 


Child Support 


1.79% 


2.00% 


-0.21%**'*' 


-0.21%*** 


Drug Abuse 


0.03% 


0.06% 


-0.03%*** 


-0.03%*** 


Texas Health Steps 


1.19% 


1.23% 


-0.04% 


-0.04% 


Choices 1 st Offense 


1.41% 


1.38% 


0.04% 


0.03% 


Choices 2nd Offense 


0.58% 


0.49% 


0.09%*** 


0.08%*** 


Choices 3rd Offense 


0.45% 


0.40% 


0.05%* 


0.05%* 


Immunization 


0.82% 


0.89% 


-0.07% 


-0.07% 


Parenting Skills 


0.07% 


0.07% 


-0.01% 


-0.01% 


Voluntary Quit 


0.04% 


0.05% 


-0.01% 


-0.01% 


Any Penalty 


8.37% 


8.49% 


-0.11% 


-0.11% 


Average Leneth of Penalties (months) 


School Attendance 


3.58 


3.70 


0.11 


0.11 


Child Support 


2.86 


2.57 


-0.28*** 


-0.26** 


Drug Abuse 


3.53 


3.70 


0.17 


-0.30 


Texas Health Steps 


2.60 


2.59 


-0.01 


-0.01 


Choices 1 st Offense 


2.06 


2.09 


0.04 


0.04 


Choices 2nd Offense 


2.42 


2.55 


0.14 


0.14 


Choices 3rd Offense 


2.73 ^ 


2.76 


0.10 


0.10 


Immunization 


3.38 


3.57 


0.18 


0.18 


Parenting Skills 


2.80 


3.04 


-0.09 


-0.09 


Voluntary Quit 


1.96 


1.67 


-0.29 


-0.29 


Any Penalty 


2.95 


2.91 


-0.04 


-0.04 


Rate of penalty-related exits 


12.88% 


13.10% 


0.22% 


0.20% 



statistically significant at 1% level 
♦♦ statistically significant at 5% level 
♦ statistically significant at 10% level 



3.1.2 Self-Sufficiency 

Table 7 shows that there were no statistically significant differences between 
participants in the experimental and control groups with respect to the percent of quarters 
during which the caretaker earned wages of any amount or the number of quarters in 
which caretakers were employed in the year following random assignment. In addition, 
no statistically significant differences were observed between participants in the 
experimental and control groups for any of the five earnings variables. 
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Table 7. Employment and Earnings for Time Limits 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Employment 




Percent of quarters during which 
caretaker had wages of any amount 


35.5% 


35.5% 


0.0% 


0.00 


Percent of caretakers who were 
employed in all four quarters in the 
year after random assignment 


6.4% 


6.7% 


-0.2% 


0.00 


Percent of caretakers who were 
employed in at least three quarters 
in the year after random 
assignment 


12.5% 


13.0% 


-0.5% 


0.00 


Earnings 




Average quarterly caretaker wages 
(including quarters of zero wages) 


$557 


$557 


$0 


$5 


Average quarterly caretaker wages 
(excluding quarters of zero wages) 


$1,569 


$1,571 


-$2 


-$5.09 


Average quarterly family wages 
earned 


$671 


$682 


-$11 


-$4.62 


Percent of quarters during which 
caretaker wages exceeded 155% of 
poverty 


0.3% 


0.2% 


0.1% 


0.05% 


Percent of quarters during which 
family earnings exceeded 1 55% of 
poverty 


0.4% 


0.3% 


0.0% 


0.00 



None of the experimental -control differences or adjusted net impacts were statistically significant 



3.1.3 Participation in Workforce Development Services 

Data on participation in the Choices program and other workforce development 
services (e.g., JTPA) were analyzed to determine whether the treatments may have led to 
an increase in participation in workforce development services among the experimental 
group. As can be observed in Table 8, no statistically significant differences in rates of 
participation in workforce development services were observed during the early months 
of the time limits experiment. 
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Table 8. Workforce Development Participation for Time Limits 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percent participating in Choices 
program 


8.14% 


8.13% 


0.01% 


0.01% 


Average hours of Choices 
participation per month 


85.84 


86.53 


-0.69 


-0.64 


Percent participating in JTPA 


0.91% 


0.93% 


-0.02% 


-0.02% 



None of the experimental-control differences or adjusted net impacts were statistically significant 



3.1.4 Subsidized Child Care 

Table 9 highlights the effects of the experiment on the use of subsidized child care 
(SCC).^’ A statistically significant but very small difference (0.25%) between 
participants in the experimental and control groups was observed in the rate of SCC 
usage at the case level monthly, with participants in the experimental group using 
subsidized child care more often than the members of the control group. However, there 
were no statistically significant differences between groups for the percentage of children 
using SCC monthly or subsidy amoimt per child-months. 



Table 9. Subsidized Child Care for Time Limits 





Post-Treatment Mean 


Experimental 

-Control 


Adjusted 

Net 




Experimental 


Control 


Difference 


Impact 


Percentage of cases using SCC 
monthly 


3.98% 


3.88% 


0.09% 


0.25%* ♦ 


Percentage of children using SCC 
monthly 


3.13% 


3.10% 


0.03% 


0.06% 


Subsidy per child-month using SCC 


$240 


$236 


S4 


-S2 



♦♦ statistically significant at 5% level 



Subsidized child care (SCC) services are offered to eligible current and former TANF-recipient families 
under a number of different programs including Choices, Transitional, and At-Risk (income-eligible) Child 
Care. In this report, all child care subsidies administered by the CCMS system or by DHS have been 
included, regardless of eligibility type or funding source. 
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3.1.5 Summary of Time Limits Results 



Very few differences between experimental and control groups in the time limits 
experiment were observed for the period of study included in this report. After analyzing 
welfare dynamics, self-sufficiency, participation in workforce development services, and 
use of subsidized child care, positive net impacts were observed only for transfers from 
TANF cash benefits to transitional benefits and the percentage of cases using subsidized 
child care each month. The magnitude of both of these impacts were quite small (0.9% 
for transfer to transitional benefits and 0.25% for cases using subsidized child care 
monthly). Approximately 13 percent of all exits for both the experimental and control 
groups occurred in the month of or the month following the assessment of a penalty for 
failure to comply with PRA provisions. 

By the end of December 1997, only 92 persons (1.4%) in Bexar County had 
exited TANF because of reaching their time limit. Due to the length of time needed for 
time limits to take effect for Tier II and Tier III recipients, more time needs to elapse 
before the impacts of this experiment can be fully measured. 

3.2 RER Choices Experiment 

From June 1996 through December 1997, 9,566 cases were assigned to participate 
in the RER Choices experiment as either experimental or control group members.^* As 
was the case for time limits and as shown in Figure 2, most of these cases were already 
receiving TANF at the beginning of the experiment. Tests of random assignment 
revealed no statistically significant differences in either the number of persons assigned 
to the experimental or control groups or in the demographic characteristics of the 
caretakers participating in the RER experiment in Choices counties (Table 10). Thus, for 
the time period included in this report, the RER Choices experiment passed all tests of 
random assignment. 



These numbers include cases assigned in Beaumont, Corpus Christi, and Odessa only. An additional 
2,640 cases were assigned in the Clint office in El Paso. As discussed earlier, Clint results are analyzed 
separately and included in Appendix B. 



Figure 2. Number of Cases Assigned to RER Choices (by month) 
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Table 10. RER Choices Experiment: 
Characteristics of Experimental and 
Control Group Members at Random Assignment 



Attribute 


Experimental 


Control 


Experimental 

-Control 

Difference 


Did not finish high 
school 


16.7% 


16.0% 


0.7% 


Male 


5.3% 


5.0% 


0.3% 


Black 


35.7% 


37.0% 


-1.3% 


Hispanic 


43.0% 


41.3% 


1.7% 


Age 


31.3 


31.3 


-0.1 


Months on TANF out of 
last 12 months 


6.9 


6.9 


-0.1 


Months employed out of 
last 12 months 


3.8 


3.7 


0.1 


Any employment in last 
12 months 


50.4% 


50.1% 


0.3% 


Total U1 earnings in last 
12 months 


$1,847 


$1,733 


$114 


In TANF UP at time of 
random assignment 


2.2% 


2.4% 


-0.2% 



None of the experimental-control differences were statistically significant 



Net impacts of the RER-Choices experiment on welfare dynamics, self- 
sufficiency, participation in workforce development services, and use of subsidized child 
care are discussed below. 

3.2.1 Welfare Dynamics 

3.2.1. 1 Welfare Exits and Recidivism 

Table 11 highlights the effects of the RER Choices experiment on welfare 
dynamics through December 1997. Statistically significant differences in the impacts of 
the experiment on the various kinds of exit rates and recidivism were observed, but in 
spite of these differences, the overall effects on time spent on TANF and total caretaker 
exits were rather small. This seeming inconsistency came about because two of the three 
individual exit rate effects and the recidivism effect tended to reduce the experimental 
caseload relative to the control caseload, whereas one of the exit rates tended to increase 
the experimental caseload. Specifically, experimental group members were more likely 
to exit to medical assistance-only or transitional benefits-only status, and less likely to 
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experience recidivism, whereas experimental group members were less likely to exit to 
payee-only and case name-only status. 

Experimental group members spent a smaller proportion of the total possible 
months on TANF than control group members. About 61 percent of caretakers in the 
experimental group exited from TANF compared to 58 percent of those in the control 
group. As of December 1997, 9.1 percent of the experimental group members transferred 
to medical assistance-only and 10.6 percent to transitional benefits-only , compared to 7.2 
percent and 9.1 percent for their control group counterparts. However, only 7.8 percent 
of experimental group members transferred to payee-only status compared to 12.8 
percent of control group members. The lower rates of transfer to payee-only status 
probably occurred because experimental group members who fail to comply to PRA 
provisions receive a financial penalty while control group members not cooperating with 
Choices or child support are totally removed from the case. Overall, these findings 
suggest that the RER experiment is inducing a statistically significant reduction in 
welfare participation, though the magnitude of the effect is rather modest in the first 1 9 
months of the experiment. . 

Table 1 1 also highlights recidivism rates for the RER Choices experiment. 
Participants in the experimental group had a significantly lower one-year recidivism rate 
(34.6 percent) than their control group counterparts (38.7 percent) when differing 
demographic factors were taken into account. This finding indicates that the RER 
initiative was effective in reducing welfare recidivism in its first 1 9 months of operation. 
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Table 11. Welfare Dynamics for RER Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Proportion of time spent on TANF out 
of maximum possible 


55.3% 


57.1% 


-1.85%*** 


-1.7%*** 


Exits 




Total caretaker exit rate 


60.8% 


58.2% 


2.6%** 


1.1% 


Rate of transfer to medical 
assistance only status 


9.1% 


7.2% 


1.9%*** 


1.7%*** 


Rate of transfer to transitional 
benefits only status 


10.6% 


9.1% 


1.5%** 


1.4%** 


Rate of transfer to payee-only and 
case-name only status 


7.8% 


12.8% 


-5.1%*** 


-5.6%*** 


Recidivism” 










One-year recidivism rate for all 
exits 


34.6% 


38.7% 


-4.1%*** 


-4.3%*** 


Average months to recidivism for 
exits followed by recidivism 


2.39 


2.41 


-0.02 


-0.01 



statistically significant at 1% level 
** statistically significant at 5% level 



3.2.1.2 Time-Limit Inducted Exit Rates 

The implementation of time limits in the RER Choices research sites occurred in 
January 1997. Thus, by the end of December 1997, only 6 exits occurred among Tier I 
clients who reached their time limit. This accounted for 0.3 percent of all Tier I cases. 
Because only 12 months elapsed between the begiiming of the experiment and the end of 
the period included in this report, no Tier II or Tier III clients were forced to exit TANF 
due to the expiration of their time limits. More time is needed to assess the effects of 
time limits on forced welfare exits in this experiment. 



3.2.1.3 Penalty Rates 

In the RER experiment in Choices counties, experimental group members receive 
financial penalties for failure to comply with the PRA provisions outlined in Table 12, 
while control group members are not subject to such penalties. Control group members 
do receive a sanction, as they did under pre-reform rules, for failure to cooperate with 



Federal project reviewers have suggested that there may be possible bias in the recidivism estimates due 
to self-selection. Future reports will discuss the possibility of self-selection bias in more depth. 



Choices participation requirements or efforts to collect child support. If sanctioned for 
one of these reasons, the caretaker in a control group case is removed from the TANF 
cash grant, which induces a transfer to payee-only status. 

As shown in Table 12, 8.5 percent of case-months for experimental group 
members were spent in penalty status in the first nineteen months of the experiment, with 
the length of completed penalties averaging 2.7 months each. Over 1 5 percent of all exits 
for experimental group members were penalty-related.’” Such exits probably contribute 
to the higher transfers to medical assistance only and transitional benefits observed for 
experimental group members. 



Table 12. Penalties Identified for Experimental Group in RER Choices 



r^xperimental 


Percent of case-months in penalty status 


School Attendance 


1 .77% 


Child Support 


2.n% 


Drug Abuse 


0.02% 


Texas Health Steps 


1.09% 


Choices 1st Offense 


1.40% 


Choices 2nd Offense 


0.80% 


Choices 3rd Offense 


0.78% 


Immunization 


0.93% 


Parenting Skills 


0.06% 


Voluntary Quit 


0.07% 


Any Penalty 


8.50% 


Average Length of Penalties (months) 


School Attendance 


3.42 


Child Support 


2.76 


Drug Abuse 


3.00 


Texas Health Steps 


2.26 


Choices 1st Offense 


1.80 


Choices 2nd Offense 


2.23 


Choices 3rd Offense 


2.78 


Immunization 


3.16 


Parenting Skills 


2.00 


Voluntary Quit 


2.07 


Any Penalty 


2.66 


Rate of penalty-related exits 


15.24% 



For the purpose of this report, penalty-related exits are defined as exits from TANF accompanied by a 
penalty in the same month or prior month. 
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Data on sanctions for control group members were not available for this report 
and will be included in later reports. However, the higher rates of transfers to payee-only 
status observed above are probably associated with caretakers in the control group being 
removed from the TANF grant when sanctioned. 

3.2.2 Self-Sufficiency 

As Table 1 3 shows, no statistically significant differences between experimental 
and control group members were found for any of the measures of employment and 
earnings. No statistically significant differences between the groups were observed in 
employment rates, number of quarters in which caretakers were employed, average 
wages, or the percentage of quarters in which caretakers’ earnings exceeded the poverty 
level. 



Table 13. Employment and Earnings for RER Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Employment 




Percent of quarters during which 
caretaker had wages of any amount 


39.6% 


39.0% 


0.6% 


0.00 


Percent of caretakers who were 
employed in all four quarters in the 
year after random assignment 


8.0% 


8.6% 


-0.6% 


-0.01 


Percent of caretakers who were 
employed in at least three quarters 
in the year after random 
assignment 


15.2% 


14.6% 


0.6% 


0.01 


Earnings 




Average quarterly caretaker cages 
(including quarters of zero wages) 


$631 


$618 


$12 


-$8 


Average quarterly caretaker wages 
(excluding quarters of zero wages) 


$1,593 


$1,585 


$9 


-$8.80 


Average quarterly family wages 
earned 


$762 


$742 


$20 


$2.90 


Percent of quarters during which 
caretaker wages exceeded 155% of 
poverty 


0.4% 


0.4% 


0.0% 


-0.10% 


Percent of quarters during which 
family earnings exceeded 155% of 
poverty 


0.5% 


0,5% 


0.0% 


0.00 



None of the experimental-control differences were statistically significant 
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3.2.3 Participation in Workforce Deveiopment Services 



Table 14 shows workforce development participation for the RER Choices 
experiment. A smaller percent of experimental group members participated in the 
Choices program than their control group coimterparts (6.34 percent versus 7.49 percent). 
This statistically significant difference can be attributed to higher percentages of control 
group members who participated in Choices’ education (32.3 percent v. 29.9 percent) and 
job search (26.1 percent v. 24.5 percent) activities. No statistically significant differences 
between the treatment group and control group participants were found in the actual 
hours of Choices participation or in percent participating in JTPA. 



Table 14. Workforce Development Participation for RER Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percent participating in Choices 
program 


6.34% 


7.49% 


-1.15%*** 


-1.06%*** 


Average hours of Choices 
participation per month 


89.82 


91.71 


-1.90 


-1.56 


Percent participating in JTPA 


1.12% 


1.13% 


-0.01% 


0.00% 



*** statistically significant at 1% level 



3.2.4 Subsidized Child Care 

Table 15 summarizes the effects of the RER Choices experiment on the use of 
subsidized child care (SCC).^' No statistically significant differences in the rates of child 
care usage were observed at either the case-level or the child-level. On the other hand, 
the treatment group participants had larger average monthly subsidies ($199) than control 
group participants ($190), and the $9 difference was statistically significant. This finding 
is consistent with the earlier finding that experimental group members had a higher 
transfer rate from welfare to transitional benefits and a lower recidivism rate than control 
group members. With a larger number of participants exiting or transitioning from 

Subsidized child care (SCC) services are offered to eligible current and former TANF-recipient families 
under a number of different programs including Choices, Transitional, and At-Risk (income-eligible) Child 
Care. In this report, all child care subsidies administered by the CCMS system or by DHS have been 
included, regardless of eligibility type or funding source. 



welfare in the experimental group, the need to receive subsidized child care would 
increase. 



Table 15. Subsidized Child Care for RER Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percentage of cases using SCC 
monthly 


4.28% 


4.39% 


-0.11% 


-0.14% 


Percentage of children using SCC 
monthly 


3.80% 


3.94% 


-0.14%* 


-0.08% 


Subsidy per child-month using SCC 


$199 


$190 


$9*** 


$13** 



♦♦♦ statistically significant at 1% level 
♦ statistically significant at 10% level 



3.2.5 Summary of RER Choices Results 

In the first nineteen months of the RER experiment in Choices counties, 
experimental group members spent significantly less time on TANF than their 
counterparts in the control group. They also experienced significantly higher transfer 
rates to medical assistance-only status and transitional benefits-only status but a 
significantly lower transfer rate to payee-only status than their control group counterparts. 
These differing patterns contribute to the overall insignificant differences in the overall 
caretaker exit rates for the two groups with 60.8 percent of caretakers in the experimental 
group exiting welfare compared to 58.2 percent for the control group. Penalty-related 
exits comprised 1 5 percent of all exits for experimental group members. There were no 
penalty-related exits in the control group because the design of the experiment specifies 
that no penalties are to be imposed on the control group. 

Experimental group members also experienced lower rates of welfare recidivism 
in the year following exit than control group members, with 34.6 percent of experimental 
group members re-entering welfare compared 38.7 percent of control group members. 
This difference in recidivism rates was statistically significant. 

Although experimental group members left TANF and remained off the rolls at 
higher rates than control group members, there were no statistically significant 
differences in employment rates or earnings levels between the two groups. Statistically 
significant differences in Choices participation rates were observed, with participants in 



the experimental group having significantly lower Choices participation rates than control 
group members. This difference may be attributed to higher levels of participation in 
education and job search activities by control group members. 

Finally, although there were no statistically significant differences in the rates of 
child care usage between the two groups, experimental group members used a larger 
amount of child care subsidy per child-month than their counterparts in the control group, 
and this difference was statistically significant. This finding, when coupled with the 
finding of higher exits to transitional benefits, suggests that experimental group members 
are using the larger child care subsidy to assist in their transition from welfare to work. 

Taken together, these findings suggest that the combination of RER and time 
limits in the RER Choices experiment may be causing a non-trivial number of caretakers 
to leave and remain off the TANF rolls in the early months of the experiment. However, 
there is no evidence as yet that these persons are becoming employed at higher rates or 
earning more money. These measures, along with additional measures of child well- 
being, will continue to be examined in future reports. 

3.3 RER Non-Choices Experiment 

The RER experiment in Non-Choices research sites was implemented in January 
1997. By the end of December 1997, 1,358 cases were assigned to participate in the RER 
Non-Choices experiment as either experimental or control group members. As shown in 
Figure 3, most of the cases participating in this experiment during its first year of 
operation were already receiving TANF at the beginning of the experiment. 

While no statistically significant differences were observed in the number of cases 
assigned to experimental and control groups, some statistically significant differences 
exist between the demographic characteristics of the two groups (Table 16). Specifically, 
experimental group members are slightly younger than control group members and have 
earned less in the past year. Also, the control group contains a higher percentage of 
Black caretakers than the experimental group. CSHR researchers are still investigating 
possible reasons for the differences in the demographic characteristics between the two 
groups. 
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Figure 3. Number of Cases Assigned to RER Non-Choices (by month) 
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Table 16. RER Nom-Chokes Experimeiil: 
Characternstks of Experimental and 
Control Group Members at Random Assignment 



Attribute 


Experimental 


Control 


Experimental 

-Control 

Difference 


Did not finish high 
school 


15.1% 


19.1% 


-3.9% 


Male 


7.2% 


6.9% 


0.3% 


Black 


22.5% 


27.5% 


-5.0%* 


Hispanic 


45.6% 


47.1% 


-1.5% 


Age 


32.6 


33.8 


-1.2* 


Months on TANF out of 
last 12 months 


6.7 


7.0 


-0.3 


Months employed out of 
last 12 months 


3.1 


3.2 


-0.1 


Any employment in last 
12 months 


44.2% 


45.9% 


-1.7% 


Total UI earnings in last 
12 months 


$1,553 


$2,133 


-$579* 


In TANF UP at time of 
random assignment 


7.0% 


5.1% 


1.9% 



* statistically significant at 10% level 



Net impacts of the RER Non-Choices experiment on welfare dynamics, self- 
sufficiency, participation in workforce development services, and use of subsidized child 
care are discussed below. 

3.3.1 Welfare Dynamics 

3.3.1. 1 Welfare Exits and Recidivism 

Table 17 highlights the effects of the experiment on welfare dynamics for the 
RER Non-Choices experiment. 

As shown in the table, there were no statistically significant differences between 
experimental and control group members for any of the four exit measures in the RER 
Non-Choices experiment. However, control group members spent a statistically 
significant higher proportion of time on TANF out of the maximum possible than the 
experimental group participants. The inconsistency between results for the total exit rate 
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and the proportion of time spent on TANF is probably due both to the relatively short 
time period available in which to calculate welfare exits and to the small sample size of 
participants in this experiment. 

Because only one year had elapsed between the beginning of this experiment and 
the end of the period of study, it was not possible to calculate one-year recidivism rates 
for persons who exited from TANF. These results will be included in future reports. 



Table 17. Welfare Dynamics for RER Non-Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Proportion of time spent on TANF out 
of maximum possible 


64.3% 


66.6% 


-2.34%* ♦♦ 


-1.4%* 


Total caretaker exit rate 


49.9% 


48.1% 


1.8% 


-1.2% 


Rate of transfer to medical assistance 
only status 


12.2% 


10.7% 


1.5% 


0.1% 


Rate of transfer to transitional benefits 
only status 


4.3% 


4.0% 


0.3% 


0.1% 


Rate of transfer to payee-only and 
case-name only status 


8.3% 


8.8% 


-0.4% 


1.3% 


One-year recidivism rate for all exits^^ 
Average months to recidivism for exits 
followed by recidivism 











*** statistically significant at \% level 

* statistically significant* statistically significant at 10% level 



33A.2 Penalty Rates 




In the RER experiment in Non-Choices counties, experimental group members 
receive financial penalties for failure to comply with the PRA provisions outlined in 
Table 18, while control group members are not subject to such penalties. Control group 
members are sanctioned and removed from the TANF grant, as they were under pre- 
reform rules, for failure to cooperate with child support collection efforts. 

Table 18 indicates that 4.4 percent of total case-months for experimental group 
members were spent in penalty status in the first year of the experiment. The length of 
completed penalties averaged 2.6 months each, and 5.6 percent of all exits for 
experimental group members were penalty-related. The lower overall rates of penalties 

There were no recidivists in the RER Non-Choices experiment. 
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and penalty-related exits in this experiment can be attributed to the absence of Choices- 
related penalties in these Non-Choices offices. 



Table 18. Penalties Identified for Experimental Group in RER Non-Choices 



Experimental 


Percent of case-months in penalty status 


School Attendance 


1.27% 


Child Support 


2.05% 


Drug Abuse 


0.12% 


Texas Health Steps 


0.99% 


Immunization 


0.28% 


Parenting Skills 


0.14% 


Voluntary Quit 


0.17% 


Any Penalty 


4.41% 


Average Length of Penalties (months) 


School Attendance 


2.70 


Child Support 


3.00 


Drug Abuse 


5.00 


Texas Health Steps 


2.00 


Immunization 


3.00 


Parenting Skills 


3.00 


Voluntary Quit 


2.33 


Any Penalty 


2.63 


Rate of penalty-related exits 


5.67% 



3.3.2 Self-Sufficiency 

Table 19 shows that there were no statistically significant differences between 
members in the experimental and control groups for any of the employment rate 
measures. In addition, no statistically significant differences between the experimental 
and the control groups were found for any of the earnings meaisures. 
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Table 19. Employment and Earnings for RER Non-Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Employment 




Percent of quarters during which 
caretaker had wages of any amount 


34.2% 


32.1% 


2.1% 


0.00 


Percent of caretakers who were 
employed in all four quarters in the 
year after random assignment 


3.0% 


3.4% 


-0.4% 


-0.01 


Percent of caretakers who were 
employed in at least three quarters 
in the year after random 
assignment 


9.0% 


6.9% 


2.1% 


0.00 


Earnings 




Average quarterly caretaker wages 
(including quarters of zero wages) 


$510 


$461 


$49 


$51 


Average quarterly caretaker wages 
(excluding quarters of zero wages) 


$1,492 


$1,436 


$55 


$124.04 


Average quarterly family wages 
earned 


$643 


$577 


$66* 


$95.75 


Percent of quarters during which 
caretaker wages exceeded 155% of 
poverty 


0.2% 


0.1% 


0.1% 


0.17% 


Percent of quarters during which 
family earnings exceeded 155% of 
poverty 


0.3% 


0.2% 


0.1% 


0.00 



* statistically significant at 10% level 



3.3.3 Participation in Workforce Development Services 

While the Choices program is not available to participants in the RER Non- 
Choices experiment, they can participate in JTPA activities to enhance their workforce 
skills. Early results of the RER Non-Choices experiment indicate that a statistically 
significant difference exists in the JTPA participation rate, with participants in the control 
group having a higher rate than the experimental group. However, even though it is 
statistically significant, the magnitude of the difference in JTPA participation rate 
between the two groups is rather small— less than 0.5%. 
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Table 20. Workforce Development Participation for RER Non-Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percent participating in Choices 
program 


*11 




f. 




Average hours of Choices 
participation per month 








V, "> . -/ • 


Percent participating in JTPA 


0.67% 


0.95% 


-0.28%* 


-0.34%** 



*♦ statistically significant at 5% level 
♦ statistically significant at 10% level 



3.3.4 Subsidized Child Care 

Table 21 highlights the effects of the RER Non-Choices experiment on subsidized 
child care (SCC).” Results for the RER Non-Choices experiment show some differences 
between experimental and control group participants for the rate of subsidy use per child 
level and the amount of subsidy per child month. However, the statistically significant 
experimental effect disappears when differing demographic factors are taken into account 
{see adjusted net impact). 



Table 21. Subsidized Child Care for RER Non-Choices 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percentage of cases using SCC 
monthly 


0.41% 


0.26% 


0.15% 


0.25% 


Percentage of children using SCC 
monthly 


0.26% 


0.14% 


0.12%** 


0.07% 


Subsidy per child-month using SCC 


$245 


$204 


$41»** 


$69 



statistically significant at 1% level 
♦♦ statistically significant at 5% level 



Subsidized child care (SCC) services are offered to eligible current and former TANF-recipient families 
under a number of different programs including Choices, Transitional, and At-Risk (income-eligible) Child 
Care. In this report, all child care subsidies administered by the CCMS system or by DHS have been 
included, regardless of eligibility type or funding source. 



3.3.5 Summary of RER Non-Choices Results 

Welfare dynamics results for the first year of the RER experiment in Non-Choices 
counties indicate that, while experimental'group members spent less time on TANF than 
their coimterparts in the control group, no statistically significant differences were foimd 
for any of the other welfare exit measures. The inconsistency between results for the 
proportion of time spent on TANF and other welfare exit measures is probably due to the 
relatively short time period available in which to calculate welfare exits and the small 
sample size of participants in this experiment. Approximately five percent of all exits 
were penalty -related. As of December 1997, insufficient time had elapsed to calculate 
welfare recidivism rates for this experiment. 

No statistically significant differences between the experimental and control 
groups were found for employment, earnings, or any of the child care measures. While a 
statistically significant difference was foimd for JTPA participation rates, the magnitude 
of the difference was trivial. 

Primarily due to the limited time period included in this interim report and the 
small sample size in this experiment, it would be premature to infer much from the 
interim findings for this experiment. 
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Appendix A: Technical Appendix 



This appendix provides more detailed information on data sources, variable 
definitions, research methods, research limitations and tests of random assignment. 

A.1.1 Data Sources 

The following section of text provides details of the data collected from each 
supplying agency. 

A.1.1.1 Department of Human Services (DHS) Data 

The primary data source for the experiment was the DHS’ SAVERR data system, 
the main repository of client and case information. Most other DHS data systems interact 
with the SAVERR system, which has been modified to include the collection of 
additional elements of data related to the waiver evaluation. DHS has developed 
procedures to extract the necessary data from the SAVERR system for the experimental 
and control groups, and store the records on an evaluation tape which was made available 
to CSHR for analysis. This tape constituted the master file of the waiver population, and 
all other files were linked to it by client number or social security number (SSN). 

CSHR relied upon other DHS data sources as appropriate to supplement the data 
on the evaluation tape. These supplementary sources included: 

. monthly TANF client strip tapes covering the demonstration period 

• annual unduplicated TANF tapes containing history of TANF spells of receipt 

• transaction files including the disposition of TANF applications and 
recertifications; and 

• CCMS and payment data for periods when DHS was responsible for managing 
that system (before September 1996 ). 

A.1.1.2 Texas Workforce Commission (TWC) data 

TWC is the source of Choices participation administrative data. The data 
includes monthly tallies of actual hours of participation in each Choices component 
activity. Also, as the administrator for the Unemployment Insurance (UI) program, TWC 
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maintains a wage database system that contains reported employee wages by each 
quarter. The data identifies each employee by SSN, and is thus easily linked to members 
of the experimental and control groups. These data were used by CSHR researchers. 

The administration of SCC was transferred to TWC from DHS in September, 
1996. In order to answer research questions regarding the use of SCC, CSHR requested 
the CCMS and Child Care payment administrative data from TWC and merged it with 
the earlier data from DHS. Case-by-case data requested regarding SCC included, but was 
not limited to: (1) spells of SCC receipt, (2) number of children covered, and (3) costs of 
providing care. 

The Choices program is not the only source of education, training and job search 
services that are available to indigent unemployed persons. The JTPA program offers 
very similar services. Since individuals who are denied Choices participation either by 
their remoteness, or because of lack of availability of slots, may seek services through 
JTPA, CSHR researchers also collected JTPA participation data from TWC to fully 
measure participation in workforce development services. 

A.1.2 Creation of Research Data Sets 

To conduct the analysis, CSHR researchers linked and merged data files from the 
disparate data sources noted in Table 2 of the main report. The first step in pulling this 
data together was to assimilate the DHS evaluation tape and extract a listing of the SSN’s 
and client numbers of all experimental and control group participants and the dependents 
on their cases. These extracts of the identifying numbers were sent to the agencies 
providing the data to be linked to records in their files. Linked records were placed in a 
file which was transmitted to CSHR. CSHR researchers then created a relational data 
engine that ties together several relational datasets to produce flat files for analysis. The 
format of the flat files differs depending on the type of research questions (i.e., case- 
quarter and case-month). 

In some cases, confidentiality requirements or other administrative roadblocks 
precluded obtaining individual-level data, resulting in some research measures not being 
examined in this report. These unavailable data include measures on education, 
immunization and child protective service (foster care and child abuse) outcomes. CSHR 
will continue to follow-up with appropriate agencies in collecting these data. If this is not 



feasible, CSHR researchers, in consultation with DHS, will develop alternative research 
strategies. 

A.1.3 Analyzed Variables 

A.1.3.1 Welfare Dynamics 

The expression “welfare dynamics” refers to changes in the stock of active cases 
and the flow of cases into and out of active status. Since the stocks and flow of cases are 
time variant, their analysis is called “dynamic” in the mathematical sense of the word. 
Caseload is determined by the following formula; 

Caseload, = Caseload, + Entries, — Exits, (1) 

Where: 

Caseload, =Caseload at the end of month t, 

Enlriesrliev/ cases arriving during month t, and 
Exits, =Act\ve cases leaving the rolls during month t. 



Equation (1) shows that effects on caseload are functionally dependent on effects 
on entries and exits. We performed two analyses on caseload. In the first approach, we 
computed the impact of the experimental treatments on caseload by comparing the 
average number of months on welfare after random assignment for the experimental and 
comparison groups. In the second approach, we estimated the experiment’s effects on 
exit rates. The design of the experiment precludes the analysis of changes in entry rates 
from the general population. However re-entry by previous recipients (recidivism) is 
amenable to analysis and an experimental effect on recidivism was estimated. 

A. 1.3. 1.1 Months on Welfare 

The computation of the average number of months on welfare after random 
assignment is complicated by the ongoing nature of the experimental design. People who 
enter the experiment at the beginning have more of a chance to accumulate months on 
welfare than people who enter near the end. One way to overcome this complication is to 
compare the average time on welfare for month-by-month cohorts. Unfortunately, this 
approach produces a large volume of statistics. In order to summarize caseload impacts 
in a single statistic, we developed the following procedure: First, we added up the actual 
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number of months spent on welfare after random assignment for the experimental and 
control groups. Second, we added up the number of months that the participants could 
have spent on welfare if they had stayed on welfare for the entire period from random 
assignment to the end of the observation period. Third, we divided the actual time on 
welfare by the maximum possible time they could have spent on it to get a proportion. 
This proportion was compared between the experimental and control groups. If the 
experiment induced people to exit welfare and stay away, then the proportion was smaller 
for the experimental group than it was for the control group. 

This approach summarizes the entire impact of the experiments on caseload in a 
single figure for each experiment. The combined effects of changes in entries, exits and 
recidivism were summarized in this single statistic. The separate experimental effects on 
exits and recidivism were also estimated separately. 

A.1.3.1.2 Exit Rates 

Four kind of exits were studied: total exits from active TANF status, transfers 
from cash benefits to medical assistance only, transfers from cash benefits to transitional 
benefits, and transfers from certified caretaker status (cash benefits) to payee only or 
case-name-only status. The medical assistance only status is subdivided into transitional 
Medicaid and other categories of Medicaid. The definition of transitional benefits only 
status includes transitional childcare benefits and transitional Medicaid benefits. Both 
transitional Medicaid and transitional child care benefits are offered to former recipients 
who either become employed or reach their time limit for at least one year after their exit 
from welfare. Payee-only and case-name only status covers those adult recipients whose 
needs are no longer included in (dropped off from) the TANF cash grants but their 
children remain on the active cases. Dependent variables were computed for these kinds 
of exits, and adjusted net effects were estimated from them. For this analysis, exits were 
measured based on the status of the case as of the last month of the observation period, 
regardless of the date of random assignment. 

A.1.3.1.3 Recidivism Rates 



For those participants who exit from welfare during the observation period, 
recidivism is possible. It was expected that some of the experimental treatments would 



reduce the likelihood of recidivism. To test this hypothesis, we computed the percent of 
exiters who returned and the time they took to return and compared these statistics for the 
experimental and control groups. 

As with the caseload analysis, the ongoing nature of the experiment complicates 
this analysis. Participants who exited early in the experiment had a longer follow-up 
period in which to return than participants who left near the beginmng of the experiment. 
Accordingly, for the analysis of time to return, we compared month-by-month cohorts of 
exiters, and for the probability of returning, we tabulated the returns based on a one-year 
follow-up period. Only exiters who had at least a one-year follow-up period were 
tabulated, and only returns within the one year follow-up were counted. While this 
approach ignored exits made near the end of the analysis period, this loss of data was 
inevitable. 



A. 1 .3. 1 .4 Penalty and Sanction -Rates 

Penalties may be imposed on members of the experimental sample for reasons 
highlighted in the penalty tables of the main report. Sanctions may be imposed on the 
control group members as they were under the pre-welfare-reform rules. Under these old 
rules, control group members could be sanctioned for failure to cooperate with child 
support or Choices requirements. A penalty involves a reduction in the amount of the 
grant — it is like a fine. A sanction usually involves the removal of the caretaker from the 
caseload — in addition to reducing the grant amount as a fine does, a sanction involves the 
revocation of various entitlements and eligibilities of the case head. Since penalties and 
sanctions are two different methods of punishment, it is not strictly valid to compute the 
net effect of the experiment by simply computing the difference between the number of 
penalties imposed on the experimental group and the number of sanctions imposed on the 
control group in RER Choices, RER Non-Choices, and Clint. Strictly speaking, since the 
control group receives no penalties, the net effect of the experiment on penalties is simply 
the number of penalties imposed on members of the experimental group. In order to 
describe the impacts of the experiment completely, both penalties and sanctions were 
tabulated as punishments, but difference and net effect computations were omitted. In 
the Time limits experiment, both control and experimental groups receive penalties and 
they are analyzed in the usual way. 



The tabulations of penalties and sanctions includes both the number of events and 
the percent of months spent in penalty or sanction status. In order to see if penalties and 
sanctions were inducing exits among those being punished, we identified all cases in 
which an individual exited welfare while being punished or soon after having been 
punished. The proportion of these cases out of all cases were tabulated, but were not 
subjected to analysis designed to produce estimated net effects. 

A.1.3.2 Self-Sufficiency 
A.1.3.2.1 Employment 

Some limited data on income of TANF recipients is available through the 
administrative records of the DHS. However, this data covers only current recipients, 
and is based on self-reported income. Previous work in the area of welfare and 
employment has shown that UI wage data is likely to be superior to self-reported data 
from administrative welfare records; therefore, it was used to measure employment. UI 
wage data covers over 95 percent of all employment in the state of Texas. Some jobs are 
not covered, including out-of-state employment, self-employment, and most agricultural 
employment. Any underreporting due to these reasons should fall equally on both the 
experimental and control groups. 

A.1.3.2.1. 1 Percent with any employment 

In measuring employment outcomes, CSHR researchers created a variable that 
takes the value of one if the recipient earned money and zero otherwise. Taking the mean 
of this variable for a group of individuals gave the percent employed for that group. The 
difference in the rate of employment between the experimental and control groups was 
the employment impact of the experiment. 

A.1.3.2.1.2 Length of employment 

In addition to knowing whether the subject was employed, there was an interest in 
knowing if the employment was merely a temporary episode, or if it was a steady job. 
Holding a job for one or two quarters out of a year is unlikely to make the subject self- 
sufficient. Accordingly, it was desirable to analyze the duration of the periods of 
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employment. CSHR researchers developed two measures to assess the effect of the 
experiment on the length of caretakers’ employment. First, a variable taking the value of 
one was created if the caretaker was employed in all four quarters in the year after the 
random assignment. Second, a variable taking the value of one was created if the 
caretaker was employed in three quarters in the year after the random assignment. By 
comparing these measures in percentage terms, the data reflected the effect of the 
experiment on the degree of caretakers’ length of employment. 

A.1.3.2.2 Earnings 

Previous work with UI wage data has shown that a large percentage of the welfare 
population earns wages. However, the distribution of wages earned is skewed, with a 
large proportion of the earners at the low end of the wage scale and very few at the high 
end of the wage scale. Further, there are many participants with zero wages. Earnings 
were analyzed by comparing the average amount earned by case heads in the 
experimental group to the average amount earned by case heads in the comparison group. 
Since the average earnings of the case heads may differ between the two groups both 
because of differences in the number employed and the wage level earned by the 
employed case heads, two tabulations of average wages are displayed. In the first, all 
case heads were included. This tabulation gives an overall assessment in a single easily- 
understood number of the experiment’s effect on changes in the amount of money 
available to the case heads. In the second tabulation, only case heads with earnings were 
included. The second tabulation gives an assessment of whether the experiment had an 
effect on the participants’ ability to earn higher wages. 

Many of the welfare wage earners earn so little in wages that the earnings cannot 
be reasonably expected to move the earner into a state of self-sufficiency. For this 
reason, it is desirable to analyze employment using a variable that is set to one if the 
subject earned 155 percent of the official poverty level, which some authors have 
maintained is necessary to become totally self-sufficient, and was set to zero otherwise. 
Using the percent of poverty approach has the advantage that family size is a factor in 
determining whether earnings are large enough to constitute self-sufficiency. In addition 
to the analysis of the wages of the case head, a second analysis was performed on total 
family income. The family income analysis is motivated by the idea that the family 
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moves into or out of poverty and welfare dependency as a unit, rather than individual-by- 
individual. While it may be impossible for one individual in a family to bring in wages 
that are 155 percent of poverty, it might be possible to achieve this goal if more than one 
person in the family works. Family income data may be constructed by aggregating UI 
wage records for all family members. The same analytical approaches used for earnings 
of the case head were also applied to total family income. 

A. 1.3.3 Participation in Workforce Development Services 

CSHR researchers analyzed the data on both Choices participation and 
participation in other workforce development services (e.g. JTPA) to determine whether 
the experiment contributed to an increase in the use of workforce development services. 
Choices participants may be assigned to at least 1 6 different kinds of activities designed 
to help them find jobs or make themselves more employable. CSHR examined the data 
on participation to find if the TL and RER experiments affected participation patterns. 
Also, as local workforce boards take over operations, they may induce changes in the mix 
of services offered. Service emphasis may vary by local workforce area, causing 
different patterns of participation to be observed at different sites. The variables analyzed 
to detect these shifts in activity mix were activity-by-activity net effect estimations. For 
Non-Choices experiments, CSHR studied the JTPA participation rates for the experiment 
and control group members. 

A.1.3.4 Subsidized Child Care 

Subsidized child care (SCC) services are offered to eligible current and former 
TANF-recipient families under a number of different programs including Choices, 
Transitional, and At-Risk (income-eligible) Child Care. The ACT waiver demonstration, 
among other things, changed some of the regulations governing the eligibility for and 
receipt of Transitional Child Care (TCC) services for recipients who exhausted their 
time-limited benefits. CCMS and DHS’ payment data were analyzed to determine 
whether any of the demonstration treatments affected the patterns of subsidized child care 
receipt. The experiment was expected to change both the number of children in SCC, 
and the number of families that availed themselves of this benefit. Experimental effects 
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were computed for the percentage of cases using SCC monthly, the percentage of 
children using SCC monthly and the amount of subsidy per child-month using SCC. 

A.1.4 Potential Limitations of Analysis 

The following section presents short descriptions of some of the difficulties in 
executing this research. 

A.1.4.1 Time Frame 

While the ACT demonstration will be in effect from June 1996 through March 
2002, this report only covers the time period from its inception in June 1996 through 
December 1997. The final impact report is due May 2002 and CSHR researchers will 
analyze the impacts of the ACT demonstration through December 2001. Since this report 
covers only a small portion of this overall time frame, CSHR researchers believe that it is 
premature to expect many significant differences between the experimental and control 
groups. 



A. 1.4.2 Disparate Sources for Variables 

In some cases, the primary data source changed over time. One example of this 
occurred with SCC variables, in which current SCC data came from TWC and historical 
data from DHS. This dual provider situation raises the danger that observed changes in 
the variables may not have been due to an actual change in the amount of SCC, but 
merely a change in the way these variables were tracked under the two different data 
systems. 

CSHR took steps to help ensure that data from disparate sources were as 
comparable as possible, but in these situations, it was never possible to be absolutely sure 
that all observed differences were due to the experimental treatments rather than disparate 
data collection procedures. The only mitigation to this difficulty was the hope that any 
biases in the data would fall equally on the control and experimental groups. 
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A.1.4.3 Data Censoring 



Because the waiver evaluation is of finite duration, it is impossible to follow all of 
the experimental participants to the end of time to see how the waiver provisions affected 
their lives. Since the follow-up period ends for everyone at a single point in time, people 
who have been in the evaluation from the beginning have a longer follow-up period than 
people who enter near the end of the period. CSHR researchers adopted a number of 
statistical procedures to handle this data censoring problem. In measuring welfare exits, 
the statistics were reported for month-by-month cohorts. In terms of caseload, data were 
tabulated as a proportion of time spent in a particular status out of all possible 
opportunities to be in that status. In measuring welfare recidivism, late data for early- 
comers were ignored so that all observations have the same follow-up period. 

A.1.4.4 Contamination of Treatments 

In implementing the experiment, contamination of treatments may take place 
through case workers, word of mouth and news report, and other agencies’ non- 
experimental treatments. Since the primary point of contact between the government and 
the welfare client is the case worker, the success of the evaluation depends on the ability 
of the case workers to accurately administer the treatments to the clients. If a case . 
worker incorrectly interacted with a client while not aware of the client’s special status as 
a control group member, there was a possibility that the case worker may have 
inadvertently exposed the client to the experimental protocol, and this exposure could 
have affected the client’s behavior. In terms of contamination by word of mouth, control 
group members would likely behave similar to their experimental group counterparts 
because they would be made aware by word of mouth and news reports that welfare rules 
changed statewide. It was a necessary condition for the internal validity of this 
evaluation that control group members clearly understand that they were not subject to 
the new rules that were imposed statewide. Finally, the various agencies charged with 
providing social services are continually changing the way they provide services, and 
many of these changes (i.e., welfare to work grants and workforce development reform) 
could have strong impacts on the treatments in this evaluation. 
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To minimize these potential contamination problems, DHS Office of Program 
Analysis and Evaluation is conducting a process evaluation to ensure that procedures 
used in the ACT demonstration are being carried out in a consistent marmer. 

A.1.5 Tests of Random Assignment 

The waiver is being evaluated by comparing differences in outcomes for 
randomly assigned experimental and control groups. CSHR researchers expect to 
observe, in a well-done random assignment, that the measurable characteristics of the two 
groups should differ only by chance. In order to test the hypothesis that the measurable 
characteristics of the two groups differed only due to chance, researchers performed tests 
on the means of continuous variables and proportions of qualitative variables that 
described the two groups. 

One of the first characteristics examined was the proportion of participants in the 
experimental and control groups, with the expectation that equal numbers of participants 
would be found in each group. Therefore, 95% confidence intervals were constructed for 
each of the four experiments (Time Limits, RER Choices, RER Non-Choices, and Clint) 
and the results are shown in Figure A-1 through Figure 4. While all experiments 
exceeded the confidence interval during at least one month for the period covering 
August 1996 to December 1997, the number of persons assigned to experimental and 
control groups for the TL, RER Choices and RER Non-Choices experiments generally 
fell within acceptable ranges. However, the results for the Clint experiment were 
unexpected. In this experiment, the proportion of participants in the experimental and 
control groups were significantly different for an unacceptably large number of months, 
thus increasing the likelihood of significant differences in subsequent analyses. 



Figure A-1. Bounds for Time Limits 
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Figure A-2. Bounds for RER Choices 
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Figure A-3. Bounds for RER Nou-Clioices 




Figure A-4. Bounds for Clint 




Other tests of random assignment are reported in the main text of the report. 
Taken together, all of the tests of random assignment indicate that further investigation is 
needed to determine why the tests of random assignment were not successful for the RER 
experiment in Clint. Both the CSHR impact team and the DHS process team will 
continue to investigate this situation. 
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Appendix B: Additional Statistical Impacts 



B.l RER Experiment in Clint 

From June 1996 through December 1997, 25,552 cases were assigned to 
participate in the RER experiment in El Paso’s Clint office as either experimental or 
control group members. As can be seen in Figure B-1, most of these cases were already 
receiving TANF at the beginning of the experiment. 

The findings of the Clint experiment are reported separately in this appendix for 
the following reasons. While the Clint Office is in El Paso, which is a Choices county, it 
does not offer any Choices services due to remoteness. Also, the Clint experiment did 
not pass all tests of random assignment. Figure B-1 reveals that more persons were 
assigned in the experimental group than the control group during the period from 
September 1996 to January 1997 and July 1997. Tests of random assignment further 
revealed significant differences between the experimental and control group members in a 
number of demographic characteristics (Table B-1). For instance, there was a 
significantly greater number of males assigned in the experimental group than the control 
group. CSHR and DHS researchers are investigating possible reasons for these 
differences in the data.' 



' One possible reason for these differences is that experimental group members in RER are allowed to 
disregard children’s earnings and to have a higher level of resources in determining TANF eligibility. 
Also, the work history requirement and 100-hour rule do not apply for TANF-UP families. While these 
rule changes could increase the number of experimental group members in the experiment, we would 
expect to see similar patterns in all RER experiments, not just the Clint office. 
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Figure B-1. Number of Cases Assigned to Clint (by month) 




. 4. Experimental ■ Control 
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Table B-1. Test of Random Assignment for Clint 



Attribute 


Experimental 


Control 


Experimental 

-Control 

Difference* 


Did not finish high 
school 


44.6% 


41.5% 


3.2% 


Male 


9.2% 


6.5% 


2.1%** 


Black 


0.2% 


0.4% 


-0.2% 


Hispanic 


97.9% 


97.2% 


0.7% 


Age 


31.3 


31.5 


-0.2 


Months on TANF out of 
last 12 months 


6.1 


6.7 


-0.7*** 


Months employed out of 
last 12 months 


2.1 


1.7 


0.4*** 


Any employment in last 
1 2 months 


29.5% 


24.3% 


5.2%*** 


Total UI earnings in last 
12 months 


$1,297 ^ 


$969 


$328** 


In TANF UP at time of 
random assignment 


12.3% 


8.4% 


3.9%*** 



*** significant at 1% level 
significant at 5% level 
* significant at 1 0% level 



B.l.l Welfare Dynamics 

Welfare Exits and Recidivism 

Table B-2 highlights the effects of the experiment on welfare dynamics for the 
Clint office. 

Although no significant differences between the experimental and control groups 
were observed for the time spent on TANF out of maximum possible, participants in the 
experimental group had a significantly higher total exit rate and transfer rate to medical 
assistance only status than their control group counterparts, after accounting for 
differences for demographic characteristics {adjusted net impact). Also, no significant 
differences were found between the experimental and control groups on transfer rate to 
transitional benefits only status and transfer rate to payee-only status. In terms of welfare 
recidivism measures, no statistically significant differences were found between 
participants in the experimental group and the control group (see Table B-2). 
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Table B-2. Welfare Dynamics for Clint 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Proportion of time spent on TANF out 
of maximum possible 


57.8% 


59.3% 


-1.49%*** 


-0.7% 


Exit 




Total caretaker exit rate 


69.1% 


54.3% 


14.8%*** 


3.3%* 


Rate of transfer to medical 
assistance only status 


10.7% 


5.5% 


5.2%*** 


3.3%*** 


Rate of transfer to transitional 
benefits only status 


7.8% 


6.5% 


1.3% 


0.0% 


Rate of transfer to payee-only and 
case-name only status 


11.6% 


11.5% 


0.1% 


-1.0% 


Recidivism^ 




One-year recidivism rate for all 
exits 


32.3% 


34.1% 


-1.7% 


-2.2% 


Average months to recidivism for 
exits followed by recidivism 


2.31 


2.37 


-0.06 


-0.04 



♦♦♦ significant at 1% level 
♦♦ significant at 5% level 
♦ significant at 10% level 



B.l.1.2 Penalty Rates 

In the RER experiment in Clint, experimental group members receive financial 
penalties for failure to comply with the PRA provisions outlined in Table B-3 while 
control group members are not subject to such penalties. Control group members are 
sanctioned and removed from the TANF grant, however, for failure to cooperate with 
child support collection efforts. 

Table B-3 indicates that approximately 2.6 percent of total case-months for 
experimental group members were spent in penalty status in the first nineteen months of 
the experiment, with completed penalties averaging 2.6 months each. Approximately 5 
percent of all exits for experimental group members were penalty-related. As was true in 
the Non-Choices, experiment, the lower overall rates of penalties and penalty-related 
exits in this location can be attributed to the lack of Choices-related penalties in this Non- 
Choices office. 

^ Federal project reviewers have suggested that there may be possible bias in the recidivism estimates due 
to self-selection. Future reports will discuss the possibility of self-selection bias in more depth. 
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Table B-3. Penalties for Experimental Group in CMnt 





Number of 
quarters in 
penalty status 


Percent of case-months in penalty status 


School Attendance 


1.07% 


Child Support 


0.78% 


Drug Abuse 


0.00% 


Texas Health Steps 


0.35% 


Immunization 


0.41% 


Parenting Skills 


0.08% 


Voluntary Quit 


0.03% 


Any Penalty 


2.57% 


Average length of penalties (months) 


School Attendance 


3.42 


Child Support 


2.50 


Drug Abuse 


- 


Texas Health Steps 


2.22 


Immunization 


3.36 


Parenting Skills 


4.50 


Voluntary Quit 


1.33 


Any Penalty 


2.56 


Rate of penalty-related exits 


5.38% 



B.1.2 Self-Sufficiency 

Table B-4 highlights employment and earnings measures for the Clint experiment. 
As shown, no significant differences were found between the experimental and control 
group members in both the employment and earnings measures. 
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Table B-4. Employment and Earnings for Clint 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Employment 




Percent of quarters during which 
caretaker had wages of any amount 


23.4% 


22.1% 


1.3%* 


-0.01 


Percent of caretakers who were 
employed in all 4 quarters in the 
year after random assignment 


5.6% 


5.8% 


-0.2% 


-0.01 


Percent of caretakers who were 
employed in at least 3 quarters in 
the year after random assignment 


9.8% 


9.3% 


0.5% 


0.00 


Earnings 




Average quarterly caretaker wages 
(including quarters of zero wages) 


$430 


$385 


$46* ♦ 


-$8 


Average quarterly caretaker wages 
(excluding quarters of zero wages) 


$1,840 


$1,741 


$98 


-$1.97 


Average quarterly family wages 
earned 


$601 


$544 


$58** 


-$21.57 


Percent of quarters during which 
caretaker wages exceeded 155% of 
poverty 


0.2% 


0.1% ^ 


0.1% 


0.08% 


Percent of quarters during which 
family earnings exceeded 155% of 
poverty 


0.2% 


0.1% 


0.1% 


0.00 



♦♦♦ significant at 1% level 
♦♦ significant at 5% level 
♦ significant at 10% level 



B.1.3 Participation in Workforce Development Services 

Since the Clint Office does not offer Choices services due to remoteness, 
participants in the Clint experiment can participants in JTPA activities to enhance their 
workforce skills. As shown on Table B-5, no significant differences were found between 
the experimental group and the control group members in JTPA participation rate. 
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Table B-5. Workforce Develop menit Participation for Clint 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Actual hours of Choices participation 
Percent participating in Choices 
program 






i'' • ff ' 




Percent participating in JTPA 


1.07% 


1.16% 


-0.09% 


-0.04% 



♦♦♦ significant at 1% level 
♦♦ significant at 5% level 
♦ significant at 10% level 



B.1.4 Subsidized Child Care 



Table B-6 highlights the effects of the experiment on subsidized child care (SCC). 
No significant differences were found between the experimental group and the control 
group for the percent of cases using SCC monthly and the percent of children using SCC 
monthly. Participants in the experimental group received a significantly smaller child 
care subsidy amount (per child months) than participants in the control group. This 
finding does not seem consistent with the earlier finding that experimental group 
participants had a significant higher exit rate than their control group counterparts. 
CSHR will investigate possible reasons for this particular finding. 



Table B-6. Subsidized Child Care for Clint 





Post-Treatment Mean 


Experimental 

-Control 

Difference 


Adjusted 

Net 

Impact 


Experimental 


Control 


Percent of cases using SCC monthly 


1.09% 


1.14% 


-0.05% 


-0.05% 


Percent of children using SCC 
monthly 


0.85% 


0.99% 


-0.14%** 


-0.08% 


Subsidy per child-month using SCC 


$181 


$192 


-$11 


-$50*** 



significant at !% level 
significant at 5% level 
* significant at 1 0% level 
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B.1.5 Summary of Clint Results 

Since the Clint experiment data did not pass all the tests of random assignment, 
the findings reported in this section are teiltative at best. CSHR and DHS researchers are 
currently investigating possible reasons . for the differences in the demographic 
characteristics between the experimental group and control group participants. Very few 
differences between the experimental and control groups in the Clint experiment were 
observed for the period of study included in this report. Positive net impacts were 
observed only for total exits and transfer to medical assistance status while a negative 
impact was observed for the amount of child care subsidy used. These results, however, 
are tentative since the participants in the treatment group and the control group did not 
pass all tests of random assignment. CSHR is investigating the matter and will report 
further findings in future reports. 




69 



B-8 



APR-20-00 THU 09:13 AM CTR STUDY HUM RES/UT 



FAX NO, 5124710585 



P, 02 

*.<r o/ } 




U.S, Departmont of Education 
otnca of educational Rasaarch end Imprwemont (OERI) , 
National Ubtery ofetsuoation (NLe> 

Educational Resourcos Infonnation Cantor (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 

1. DOCUMENT IDENTIFICATION; 



Tide; 



Acbigving Change for texans Evaluation 



AuttKH(s): Peanna T. Schexnaydei:. Jerome A.Olson. JaniaX G. Schroader . 



, _4ao“Chee Sim 

Corpora taiSwume; 



Center for the studay of Human Resources 



Publication Oafs: 
12/98 



II. REPRODUCTION RELEASE: 

In ikdarti itaMieift* H wUair u POHdhlB Hmoly end «bnincBil miiaiUt of Manat to the oducitional eentnwflky, documnta wwwOMed In ttw 

monlMr taamd kwmal oJ IN ERIC iyti«n, n*«oiaaM In EdueaUan |WE). in iiwtHir nud* irabibti to uMii h nikreUw. rap««d^ p«fw tan. 
«n<i •todrenlc nmrila, and »oM tofoggli «ha ERIC DeeumMil Ropradtidion Sandee (EORS). CrN# li given to Ihe eourw of oech doeumont end, V 
repiodudtan lOtoaM k gniitod. ono of IN tolicMrlng nodoBi It odM to IN document 

B pomtoilon h fliirtod k reproduso wd rttodmlnili Bid Wenllliod dowment ploMd CHECK ONE of IN Wo«*iQ three opfiooe «nd i|gn «l IN bottom 
of Bm page. 



TM wnpe* ^ 

b Mi 1 iaomiit 


Dit aanpte ilte MMi bMow b» 
MbMd b «l UvMSA 


IM tMtRli Mete Nban Mw Mi N 
Msid b « iMl MMMlI 


P0VdSSlOlltpft£PROOUCEANO 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




PERMISSION TO reproduce AND 
DISSEMINATE TNIS MATERIAL tH 

microfiche, and in electronic media 

FOR ERIC COLLECTION SUBSCR18ERS ONLY, 
HAS BEEN GRANTED GY 




PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICNE ONLY HAS BEEN OfWfm> BY 






















TO t>lE educational RESOURCES 
INrORUATlON center (ERiC) 




TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) ■' 




TO TNG EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERR) 


1' 




2A 




2B 


tiviM 

T 


Uvtl2A 

] 


LanISB 

r 


□ 


□ 


□ 






hm lb liMbS 


wM »iv inbnlkA* W i*Ar El^ 


■rd tfuanMarin b iNaoAdM ind bMfewiiB madia 

far ElUe flMM oAaaian MUob«« 


bH 4 iimHhbIimi bAdgHMb oNy 



Dinmod *■ to moM a Mkdto rnmUto rapeadM eiaiy o«mu. 

( ondMliin to noreaiee e ti«M, M M Ov a dwdto. amiMta •« to to><e**to M U«d 1 . 



Sign 
hero,** 
p/ease 



; fienhy gnni to Mo fidooMibW Mbrnufibry CMtf <wn«xdii4Vi fmrmtoahn to Mprodoca snd (Mi dooimoit 

6& fntSottod gDovg. Rsptwfuetm Rom the ERiC flriorpfitfx fir modlB hypnfsom oihjr (htn £RtC and H$ $ystom 

conmefors ftornj^ eopyrtgto hoto$r. EMeepdontomEd^tornon^nStfwprvA^iion byOirtftoMBfHicOmwytooEgonetos 

to iMtisfy k^xn^lfion c/ to tirSfiOftpb to toqutoHs. 

/ fi ) r / / / 




T. King 

Director 


3001 X-ake Austin Blvd, Ste 3.200 
Austin, TX 78703-^204 


^TT, 47 1-7891 r"51?/47i_OSfi3 


, ^Kan|(?uts. cc.utf>W;i<« f>Att f\fi to 



eO/30 'd 



098ld6SH9 'ON XVd 



nS0/3J,30 HH Et'iiO OHl 00-El -3dy 





